James Shanteau

James Shanteau is a leading thinker on the nature of expertise, presented through research that challenges the assumption that specialized knowledge reliably produces sound judgment. His work shows that experts can disagree with one another, contradict their own earlier assessments, and misjudge how likely they are to be right—making expertise alone an uncertain guide to prediction.

Research on Expertise

Shanteau devoted substantial effort to developing a method for estimating how expert a person is. Yet he also warned that experts’ decisions could be seriously flawed. The tension is important: expertise can be studied and assessed, but possessing it does not guarantee dependable judgment.

The evidence he recounts concerns both agreement between specialists and consistency within an individual expert’s assessments. Across fields including stock picking, livestock judging, and clinical psychology, agreement between experts fell below 50 percent. Specialists in these areas were therefore as likely to disagree as to agree. Medical pathologists, moreover, showed an internal consistency score of just 0.5 in one study: when presented with the same evidence, a pathologist gave a different opinion half the time.

Calibration and Forecasting

The problem extends beyond disagreement. Experts, like people generally, routinely overestimate the likelihood that their judgments are correct. The book places Shanteau’s findings within a wider account of overconfidence, citing surveys in which physicians, nurses, lawyers, engineers, entrepreneurs, and investment bankers believed they knew more than they did. A study of foreign-exchange traders found that they overestimated the accuracy of their predictions 70 percent of the time.

This distinction between being wrong and knowing how uncertain one is sharpens the critique: forecasters may fail not only to predict accurately but also to recognize the limits of their own predictions. In the account presented, expert bridge players and weather forecasters stand out as the exceptions whose judgments are routinely well calibrated; when weather forecasters assign a 30 percent chance of rain, it rains on about 30 percent of those days.

Role in the Argument About Collective Judgment

Shanteau’s research contributes to the book’s challenge to the prestige accorded to singular experts. The argument is not that individual skill is irrelevant, but that expertise beyond a minimal level offers little value in forecasting change and that a lone expert’s claims of accuracy have no practical value without reliable evidence. Shanteau’s findings give this argument a concrete basis: specialists’ judgments may be inconsistent across people, unstable within the same person, and poorly calibrated.

Against that weakness, the book describes wise crowds as depending on diversity of opinion, independent judgment, decentralization, and a mechanism for aggregation. A diverse group can combine different information while allowing individual errors to cancel in the aggregate. Shanteau’s account thus helps explain why expert status by itself cannot substitute for a process that draws on dispersed judgments and tests them collectively.